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n 


A  Decade  Ago... 

ASC 


DoD  5000.2,  Part  6-D,  Computer  Resources 


n  Computer  Resources  Life-Cycle  Management  Plan  (CRLCMP), 
Integrated  system  development,  Software  metrics,  Software  test 
management,  Ada  language  policy,  Software  engineering  practices 


n  Air  Force  Regulation  (AFR)  800-14,  Life  Cycle 
Management  of  Computer  Resources  in  Systems 


n  AFMC  Pamphlets 

n  Software  IV&V,  Software  Risk  Abatement,  Review  of  Software 
Requirements  and  Interface  Requirements  Specifications,  Software 
Management  Indicators,  Software  Quality  Measurement,  Software 
Development  Capability  Assessment 


n  SAF/AQ  Memos 

n  Software  Engineering,  Software  Maturity  Assessment,  Ada, 

Metrics,  Software  Estimating,  Software  Reuse,  Best  Practices,  Use 
of  Software  Development  Capability  Evaluation  in  Source 
Selection,  Etc. 
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A  Decade  Ago...(Cont.) 

ASC 


n  Development  standards 

n  DOD-STD-21 67/21 68,  MIL-STD-498,  MIL-STD-1803 
n  MIL-STD-882,  MIL-STD-490,  MIL-STD-499,  DOD-STD-1521 


n  Senior  software  engineer  in  each  program  office, 
supported  with  additional  help,  as  necessary 

n  Depending  on  magnitude  of  software  development  effort,  program 
phase,  etc. 

n  Air  Force  Systems  Acquisition  School  training 

n  Computer  Resources  Acquisition  Course  (CRAC) 


In  spite  of  all  this,  success  was  not  guaranteed... 
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Today... 

ASC 

n  Limited  policy  /  guidance  specific  to  the  acquisition 
of  software  intensive  systems 

n  Almost  none  of  it  mandatory 

n  No  standard  way  of  doing  business 

n  Processes  across  the  acquisition  enterprise  have  diverged 
n  Decreasing  oversight  /  insight 

n  Lack  of  appreciation  for  process 

n  Demands  for  reduced  cycle  time 

n  Training  available  through  SAM  courses 

n  Data  indicates  limited  exposure 

n  Aging  and  diminishing  workforce 

n  10  year  gap  for  new  hires 
n  Acquisition  workforce  being  rapidly  downsized 
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ASC  Workload  and  EN  Staffing 

ASC 
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The  Message(s)  to  PMs... 
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A  Sample  of  Findings  from 
ASC  Program  Reviews 


-  ASC 

n  Incompatible  /  optimistic  performance,  effort,  and 
schedule  baselines 

n  Deficiencies  in  requirements  management 

n  Inadequate  risk  identification  and  management 

n  Processes  set  aside  due  to  program  pressures 

n  Planned  reuse  not  achieved 

n  Program  staffing  problems 

n  Failure  to  identify  and  react  to  problems 

n  Inadequate  program  office  insight 

n  Labs  not  fully  capable  or  not  in  place  when  needed 

n  Fixed  price  development  contracts  with  uncertain 
requirements  or  other  significant  risks 
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An  Issue  Seen  Too  Frequently... 

-  ASC 


Inability  to  establish  compatible  effort,  schedule, 

and  performance  baselines 


n  Why? 

n  Programs  come  with  defined  cost,  schedule,  and  performance 
baselines,  often  (optimistically)  determined  without  adequate 
insight  into  what  actually  needs  to  be  done 

n  All  participants  challenged  to  reduce  cycle  times,  take  risks,  etc. 

n  Requirements  are  not  fully  defined  /  stable 

n  Difficult  to  estimate  the  size  of  a  software  development  effort  for 
unprecedented  systems  or  where  requirements  are  not 
complete 

n  Hence,  difficult  to  estimate  development  effort  and  schedule 

n  History  indicates  software  size  estimate  grows  significantly 
during  development 
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Embedded  Software  Size  Growth 


ASC 
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Embedded  Software  Schedule  Growth 

^ -  ASC 
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Plan  for  Improvement 


ASC 

n  ASC/EN  initiative  to  document  and  improve 
systems  engineering  processes 

n  Identify,  define,  &  document  technical  processes 
n  Combine  and  simplify  current  processes 
n  Fully  integrate  internal  processes 
n  Focus  on  government  responsibilities 

n  Deploy 

n  Training,  guidance,  and  monitoring 

n  Also  considering  independent  look 

n  Validation  of  selected  processes  by  outside 
organization 
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Software  Acquisition  Approach 


ASC 


n  Document  critical  processes 

n  Enterprise  support  activities 

n  Program-level  activities  (planning  and 
execution) 

n  Develop  training 

n  Target  to  all  who  need  to  know  -  not  just 
organic  engineering 

n  Deploy  and  monitor  application  of 
processes 
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Software  Acquisition  Approach 

(Cont.) 

>  ASC 


n  Organization 

n  Enterprise  (Center  Level)  Support  (3) 
n  Acquisition  Program  Planning  Processes  (4) 
n  Acquisition  Program  Execution  Processes  (14) 


n  Key  practices  addressed  in  acquisition  strategy 

n  Each  process  documented  with  brief 
description 

n  Purpose  n  Outputs  /  Products 

n  Roles  and  Responsibilities  n  Available  Tools  /  Techniques 
n  Key  steps  n  Potential  Problem  Areas  /  Pitfalls 

n  Inputs  n  Lessons  Learned 


n  Appendices  with  additional  detail  as  needed 
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Key  Software  Acquisition  Practices 

(To  Be  Addressed  in  Acquisition  Strategy) 

n  Establish  Realistic  and  Compatible  Program 
Baselines 

n  Provide  System  Development  and  Demonstration 
(SDD)  Phase  Source  Selection  Support 

n  Identify  and  Manage  Computer  System  and 
Software  Risks 

n  Establish  and  Manage  Software  Requirements 

n  Accommodate  High-Assurance  Systems 

n  Ensure  Application  of  Mature  Development 
Processes 

n  Maintain  Technical  Insight 
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Enterprise  Support 


n  Provide  Advice  and  Counsel 

n  Pre-Acquisition  Strategy  Panel  (ASP)  Support 
n  Source  Selection  Consultation  and  Advice 
n  Ensure  achievable  baselines 
n  Program  Execution 
n  Independent  Reviews 


ASC 


n  Manage  Software  Acquisition  Training  and  Experience 

n  Software  Acquisition  Engineering  Training  (Guidebook) 
n  Software  Estimation  Training 
n  Other  special-topic  training  as  required 


n  Collect  and  Disseminate  Lessons  Learned 

n  Collect  lessons  learned  at  project/build  completion 
n  Establish  and  maintain  lessons  learned  database 
n  Disseminate  lessons  learned  through  briefings,  training,  etc. 
n  Implement  needed  process  improvements 
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Acquisition  Program  Planning 


3.1  Develop  Software  Acquisition  Strategy 


ASC 


n  Develop  program  approach  to  key  software  acquisition  practices 
n  Get  informal,  independent  review  prior  to  issuing  RFP 


3.2  Establish  Realistic,  Compatible  Program  Baselines 

n  Estimate  software  development  size  (factoring  in  growth) 
n  Estimate  software  development  effort  and  schedule 
n  Develop  realistic  estimate  that  balances  risk,  cycle  time,  etc. 


3.3  Support  Request  for  Proposal  (RFP)  Preparation 

n  Provide  key  software  considerations  for  RFP  Sections  L  and  M 
n  Solicit  and  evaluate  software  size,  effort,  and  schedule  estimates 
n  Solicit  software  development  process  documentation 


3.4  Provide  Source  Selection  Support 

n  Evaluate  developer  capability 
n  Evaluate  proposed  development  processes 
n  Evaluate  proposed  development  plan 
n  Assess  compatibility  of  processes,  plan,  effort,  and  schedule 
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Program  Level  Execution 

ASC 


4.1  Identify  and  Manage  Software-Related  Risks 

n  Ensure  effective  risk  management  process  is  in  place 
n  Ensure  all  software-related  risks  are  identified  and  managed 


4.2  Establish  and  Manage  Software  Requirements 

n  Ensure  software  requirements  are  defined,  complete,  verified, 
consistent  and  traceable 


4.3  Address  Training  System  Concurrency 
Requirements 

n  Provide  for  the  most  efficient  method  to  meet  training  system 
concurrency  requirements 

4.4  Establish  Software  Build  Plan 

n  Ensure  there  is  a  plan  to  define,  develop,  integrate,  and  deliver 
software  increments  in  response  to  system  requirements 
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Program  Level  Execution  (Cont.) 

ASC 


4.5  Accommodate  Application  and  Sustainment  of  Non- 

Developmental  Software  (NDS) 

n  Address  COTS  and  other  NDS  integration 

4.6  Accommodate  Security  Certification  &  Accreditation 

(C&A) 

n  Ensure  that  confidentiality,  integrity,  and  availability  is  maintained 
throughout  the  life-cycle  of  the  system 

n  Preclude  compromise,  exploitation,  sabotage,  and  intentional 
damage  and  destruction 

4.7  Accommodate  Safety-Critical  and  High-Assurance 

Systems 

n  Define  the  process,  including  what  is  expected  of  the  developer, 
to  specify,  design,  develop,  integrate,  and  verify  flight-critical  and 
safety-critical  systems 
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Program  Level  Execution  (Cont.) 

ASC 


4.8  Establish  System  /  Software  Engineering 

Environment  (S/SEE)  and  Development  and 

Integration  Laboratories 

n  Ensure  development,  integration,  and  verification  environment 
requirements  are  fully  defined 

n  Ensure  environments  are  in  place  when  needed  and  can 
provide  the  required  throughput 

4.9  Ensure  Application  of  Mature  Development 

Processes 

n  Assess  developer  team  process  capability  prior  to  contract 
award  to  identify  strengths,  weaknesses,  and  risks 

n  Support  disciplined  application  of  processes  throughout  the 
development  effort 
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Program  Level  Execution  (Cont.) 

ASC 


4.10  Maintain  Technical  Insight  and  Resolve 

Development  Issues 

n  Implement  effective  means  of  communication  on  program 
status  and  issues 

n  Take  corrective  action  when  necessary 

4.11  Establish  Software  Product  Engineering  Data 

n  Ensure  the  minimum  set  of  engineering  data  and 

documentation  required  for  the  weapon  system  software  is 
developed,  acquired  (or  escrowed),  and  maintained 

4.12  Conduct  /  Support  Technical  Reviews 

n  Determine  the  types  of  reviews  to  be  accomplished  and  the  role 
of  the  acquisition  organization 

n  Establish  relevant  entry  and  exit  criteria 
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Program  Level  Execution  (Cont.) 

ASC 


4.13  Plan  for  Post  Deployment  Software  Support 

n  Identify  source  of  support  for  all  software  elements 
n  Determine  expected  rates  of  change  and  expected  workload 
n  Establish  required  support  resources  and  facilities 


4.14  Identify  and  Collect  Lessons  Learned 

n  Survey  project  participants 
n  Collect  objective  data 
n  Share  the  results 
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Process  Development  Schedule 


n  Complete  guidebook  draft 

n  Complete  coordination  and 
review  by  SPOs  and 
AFMC  SISSG  members 

n  Publish  guidebook  Version  1 

n  Complete  development  of 
guidebook  training 


ASC 

January  2003 
February  2003 

February  2003 
March  2003 
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Next  Steps 


n  Consider  extending  scope  to  AFMC 

n  Address  concerns  of  other  domains 
n  Add  sustainment  processes 


n  Get  leadership  buy-in 

n  Work  with  Air  Force  Institute  of  Technology 
(AFIT)  to  enhance  training 

n  Software  Professional  Development  Program  (SPDP) 
n  Acquisition-specific  training 

n  Address  Section  804  requirements 

n  Incorporate  improvements  identified  by 
independent  process  validation  activities 
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Conclusions 


n  We  understand  the  issues  and  are  taking 
positive  steps  to  set  programs  up  to 
succeed 

n  Revitalizing  our  processes  is  a  crucial 
first  step 

n  We  can't  solve  the  problem  by  ourselves 

n  Balance  risk  and  credibility 
n  Support  disciplined  application  of  processes 
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